The proliferation of mobile devices having cameras has increased the need for location recognition applications that can recognize an image taken from the mobile device's camera. Since a mobile device has limited storage capacity and computational resources, the task of visual location recognition is often performed remotely. The mobile device may upload an image taken from the mobile device to a server. The server matches the image with similar images stored on the server. The stored images are annotated with geographic data pertaining to the location of the image. The server downloads the location of the image to the mobile device. However, the mobile device may have limited bandwidth to communicate with the server. The latency involved in transmitting images and data between the mobile device and the server may be a bottleneck for the mobile device to perform the location recognition in real time.